The present invention relates generally to the design of a video surveillance system rooted in the Gaussian mixture modeling for background subtraction-based applications. More particularly, the present invention relates to a novel computational scheme of two-type learning rate control for the Gaussian mixture modeling, wherein high-level feedbacks of pixel properties are applied to the adaptive adjustments of the learning rates.
For video surveillance using static camera, background subtraction is often regarded as an effective and efficient method for differentiating foreground objects from a background scene. The performance of background subtraction highly depends on how a background scene is modeled. Ideally, an ideal design of background modeling should be able to tolerate various background variations without losing the sensitivity in detecting abnormal foreground objects. However, the tradeoff between model robustness and model sensitivity is commonly encountered in practice and is hard to be balanced within a single background modeling framework.
Among various background modeling approaches, the Gaussian mixture modeling (GMM), proposed by C. Stauffer and W. E. L. Grimson, Adaptive background mixture models for real-time tracking, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Vol. 2, pp. 246-252, 1999, is known to be effective in sustaining background variations, e.g., waving trees, due to its use of multiple buffers to memorize scene states. It is hence widely adopted as a base framework in many later developments, e.g., M. Harville, Segmenting Video Input Using High-Level Feedback, U.S. Pat. No. 6,999,620 B1, February 2006, and D.-S. Lee, Adaptive Mixture Learning in a Dynamic System, U.S. Pat. No. 7,103,584 B2, September 2006. However, the GMM often suffers from the tradeoff between model robustness to background changes and model sensitivity to foreground 20 abnormalities, abbreviated as R-S tradeoff. For instance, a Gaussian mixture model being tuned to tolerate quick changes in background may also adapt itself to stationary objects, e.g., unattended bags left by passengers, too quickly to issue reliable alarms. The lack of a simple and flexible way to manage the R-S tradeoff for various scenarios motivates this invention.
In the GMM formulations, every image pixel, regardless of its intensity being changing or not, is given the same setting of learning rates in background model estimation, which is inefficient in managing the R-S tradeoff. (The definition of learning rate is inherited from C. Stauffer and W. E. L. Grimson, Adaptive background mixture models for real-time tracking, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Vol. 2, pp. 246-252, 1999, and will be specifically defined later.) Considering a pixel of background that was just uncovered from occlusion of a moving object, the corresponding Gaussian mixture model for this pixel should be updated in a slower pace than that for a stable background pixel, to prevent false inclusion of moving shadows or motion blurs into background. Nonetheless, in the original GMM formulations, an identical learning rate setting is applied to all image pixels, leaving no space for tuning the background adaptation speeds for this case. We therefore highlight the importance of adaptive learning rates control in space and in time, and propose a new video surveillance system with a novel learning rate control scheme for the GMM.
In the present invention related to a new learning rate control scheme for the GMM, high-level feedbacks of pixel properties are applies to adaptive adjustments of the learning rates in space and in time. Despite that the idea of adopting high-level feedbacks, such as foreground pixel type, in background modeling is not new, e.g., M. Harville, Segmenting Video Input Using High-Level Feedback, U.S. Pat. No. 6,999,620 B1, February 2006, to the best of our knowledge, the proposed learning rate control scheme is the first to utilize two types of learning rate controls for enhancing the model estimation accuracy and regularizing the R-S tradeoff simultaneously. In the present invention, high-level feedbacks are applied only to the learning rate control related to the R-S tradeoff, but not to the other related to the model estimation accuracy, which leads to a unique design of a robust surveillance system.